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ESTIMATION OF VARIANCE BY A RECURSIVE EQUATION* 

By M. Melvin Bruce 
Langley Research Center 

SUMMARY 

A recursive equation is presented for the purpose of estimation of the variance of a 
sequence of independent random numbers. The use of this recursive equation makes it 
possible to perform a running estimate of the variance as the samples are received 
sequentially. An analysis of the recursive equation is included to show that it gives an 
asymptotically unbiased estimate of the variance. The variance of the estimated vari- 
ance is derived for the special case of random numbers with a Gaussian probability 
distribution. 


INTRODUCTION 

In an investigation of an adaptive binary detector, a need was discovered for a 
running estimate of the variance of a received sequence of random numbers. This esti- 
mate was required to be updated as each number was received. Therefore, the method 
of implementation of the estimation technique must be simple enough to keep the compu- 
tation time to a minimum. It was also required that the quantity of received samples not 
be a factor in the estimation technique since the system may be required to operate on a 
very long sequence of random numbers in which the number of samples could approach 
infinity. 

A search of the literature revealed no appropriate estimation techniques. J.C. Dale 
(ref. 1) discusses estimation of the variance by a sum of squares, but his method requires 
that the number of samples to be handled be known. Sliding "window" methods are also 
available, but the sliding "window" method requires that large numbers of previous sam- 
ples be labeled and stored in a memory. 

This report discusses a recursive equation that gives an asymptotically unbiased 
estimate (ref. 2) of the variance of a sequence of random numbers provided that the 

*The information presented herein was included in a dissertation entitled "A Feasi- 
bility Study of an Adaptive Binary Detector" which was offered in partial fulfillment of the 
requirements for the degree of Doctor of Science in Electrical Engineering, University of 
Virginia, Charlottesville, Virginia, August 1968. 1 



random numbers in the sequence are independent. The recursive equation is moderately 
simple to implement, has a potentially low computation time, requires a very small 
amount of data storage, and is not concerned with the number of samples to be processed. 

An analysis of the method of estimation of the variance is included to show that the 
estimation is asymptotically unbiased. The accuracy of the estimation method is investi- 
gated by deriving the asymptotic value of the variance of the estimated variance for the 
special case of random numbers having a Gaussian probability distribution. 

SYMBOLS 

a mean of function being sampled 

A factor used in estimation of mean 

B factor used in estimation of variance 

C,C2,M,P,Q constants 

expectation, average, or mean of argument given within braces 
transfer function of system 
dummy variables 
general term for mean 
complex frequency 
partial sum 
sampling period 
time constant 
general term for variance 
variance of argument within braces 
kth input data sample 
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kth estimate of mean 


*k 

X(z) z -transform of xjj 

z variable associated with z -transform, z = e sT 

a estimation factor (from ref. 3) 

<j2 variance of function being sampled 

6^2 kth estimate of variance 

p(x|y) conditional probability of x with y given 

ESTIMATION OF THE VARIANCE 

A recursive equation can be used to estimate variance. The justification of the 
equation follows the discussion of the recursive equation. The equation used for estima- 
tion of the variance is 


d k 2 = B & k _! 2 + i -^( x k - %) 2 ( 1 ) 

where 0 < B < 1.0. Equation (1) requires knowledge of the mean of the input data. If the 
mean is known, it is used in place of x k in the equation. If the mean is not known, it 
must also be estimated. For the purposes of this investigation, it is assumed that the 
mean will also be estimated by a recursive equation. The equation used for x k is 

*k = Ax k -i + (1 - A)x k (2) 

where 0 < A ^ 1.0. Brown (ref. 3) has shown that equation (2) gives an asymptotically 
unbiased estimate of the mean of x^. He has shown that the asymptotic variance of the 
estimated mean is 

lim v(x k \ zzi-^CT 2 (3) 

k— °o l K J 1 + A 

where is the actual variance of the input samples. An analysis of the recursive 
method of estimation of the mean is presented in appendix A. 

Both equations (1) and (2) must have an initial guess, ct q 2 or x Q , in order to. 
begin operation. The equation then calculates the first estimate, 0^2 or xq, from the 
initial guess and the value of the first input sample. The process is continued as more 


3 



input samples are received. The constant C in equation (1) is required to remove the 
bias in this estimation technique. 

If is replaced by equation (2), a more usable form of equation (1) is obtained: 



1 - B 


(x k - Ax k _i - x k + Ax k )" 


= B& k-1 2 




( 4 ) 


This equation for & k 2 leads to faster calculation than equation (1) since it employs x k _i 
instead of x k . This calculation can be performed in parallel with the kth estimate of the 
mean instead of after x k is calculated. 

As was done in the case of the estimate of the mean, the mean and variance of the 
estimate of the variance are determined. The constant C is selected to force the mean 
of the estimate of the variance to converge to the actual variance of the data being sam- 
pled. The variance of the estimate serves as an indication of the average error of the 
estimate. 


The mean of the estimated variance is calculated for several values of k. Enough 
terms are used to recognize the series being generated. The general expression is then 
written, and the limiting value is determined. Thus, 


&i 2 = B& 0 2 + a2 ( 1 _~ - B ) ( Xl - x Q ) 2 


and 


2l. R a^ A 2 (l-B) E r 2 _ 


E( Oi > = Bo “ + 


E i x r - 2x i x o + x c 


= B&o 2 + A2(1 c B) fc( x l 2 ) - 2 x 0 e( X i | + x Q 2 ^ 


= B& o 2 + B) ( a2 + 02 ■ 2a *0 + x o 2 ) 

= B &o 2 + A2 % : + ( a - x o) 2 
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The same techniques are used to calculate E which gives 

a 2 2 = B Uj 2 + A (1 c ~ B '( x 2 " x l) 2 

= b[bu 0 2 + ^-^( Xl - x G ) 2 ] + %^[x 2 - Ax q - (1 - A) X1 ] 2 

= B 2 a D 2 + A2B( c " B ~( x l 2 " 2x l x o + x o 2 ) + — -^-^[x 2 2 + A 2 x q 2 
+ (1 - A) 2 xj 2 - 2Ax 2 x 0 - 2(1 - A)xix 2 + 2A(1 - A)x 1 x 0 J 


and 


{» 2 2 } - + ^ 


A2(l - B) 


e|x 2 2 J + A 2 x 0 2 + (1 - A) 2 Ejx 1 2 j - 2 Ax 0 e|x 2 | 


- 2(1 - A)E jx lX2 J + 2A(1 - A)x q E jxjj 

The data samples are considered to be independent; thus, 

E jxjXjj = E jx^E^Xjj 

This relation yields 


E 


|& 2 ^ = B 2 & 0 2 + A2]B( ^ — B) (a 2 + ° 2 - 2ax Q + i Q 2 ) 


A 2 (l - B) 


i 2 + a 2 + A 2 x q 2 + (1 - A) 2 (a 2 + cr 2 ) 


- 2Aax 0 - 2(1 - A) a 2 + 2A(1 - A)ax Q 


(i^j) 


= B 2 ct q 2 + A2 ^ - B ) (1 + B)^ + (1 - A) 2 o 2 + (A 2 + B)(a - x^ 

If these same techniques are used, the mean value of & 2 2 and 04 2 can be determined: 
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eTo^ 2 '} = B 3 & 0 2 + ^ (l + B + B 2 )a 2 + (l + A 2 + b)(1 - A) 2 *? 2 


+ (a 4 + A 2 B + B 2 ) (a - x o y 

E jd 4 2 J = B 4 & q 2 + ^ 2 ^ c ~ - B - j(l + B + B 2 + B^a 2 + (l + A 2 + A 4 ) 

+ B(l + A 2 ) + B 2 (1 - A) 2 a 2 + (A 6 + A 4 B + A 2 B 2 + B 3 )(a - x Q x2 


From these four mean values it is possible to recognize the general term of this series 
as 

k-1 k-1 

^ B 1 + (a - x Q ) 2 ^ A 2 ^B k " 4- ^ 
i=0 i=0 


Ej&k 2 } = B k a 0 2 + ^ (1 C ~ B) 


k-2 / k-2-j 

(1 - A^o 2 ^ IBJ ^ A 21 ' 

3=0 \ i=0 


(5) 


The next problem is to find the value of E< o^ 2 > as k approaches infinity. Since |B| 
is less than 1.0, 


and 


lim B k & 2 = 0 

k-oo 


k-1 

lim c 2 Y B 1 = cr 2 /— 

k-oo {-> V 1 - 

1=0 


B 


The limiting value of the second term within the brackets in equation (5) is found from 
k-1 

(a - x G ) 2 ^ A 2i B k_1_i = (a - x 0 ) 2 (B k-1 + A 2 B k “ 2 + A 4 B k " 3 + . . . + A 2k “ 2 ) 
i=0 


It is known that 
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0 < A < 1 
0 < B < 1 


Let 

A < C 2 < 1 
B < C 2 < 1 

If C 2 is substituted for A and B in this series, the resulting series is greater term 
by term than the original series involving A and B. If the limiting value of the series 
of C 2 is shown to approach zero, the limiting value of the series of A and B must 
also approach zero. Thus, 


k-1 


c 2 2 i c 2 


k-l-i _ 


c 2 * 


i=0 


k-1 

l C 2 

i=0 


This series is a truncated geometric series whose partial sum s^ (ref. 4) is 


s k 


, k-1 1 ~ c 2 
2 1 - C 


C 2 k-1 - c 2 2k - 1 


1 - C 2 


Since C 2 < 1, 


lim Sj£ = 0 
k — 00 


Therefore, 

k-1 

lim ^a - x Q ) 2 ^ A 2 iB k ”^ _ * = 0 
k ~°° i=0 


The last term in equation (5) is 


k-2 / k-2-j \ 

(1 - A)2o2 £ (bJ J A2i) = (1 - A)2a2f(l + A 2 + A 4 + . . . + A 2 ^ 4 ) 
j=0 \ i=0 / L 

+ b(i + A 2 + A 4 + . . . + A 2k " 6 ) 

+ B 2 (l + A 2 + a 4 + . . . + A 2k “8) + . . . + B k-3 (l + A 2 ) + B k-2 J 
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The limiting value is 


•••) 


k-2 / k-2-j \ 

lim (1 - A^o 2 ^ p ^ A 2i J = (1 - A) 2 ^ + A 2 + A 4 + . . .)(l + B + B 2 + . . .) 

k ~°° j=0 \ i=0 / 

(1 - A) 2 ct 2 
(l - A 2 )(l - B) 

(1 - A)*? 2 

" (1 + A)(l - B) ' 

These expressions are inserted into the equation for E j^k^j ( ec l* (*>)) in order to 
find the limit as k approaches infinity. Thus, 


lim E< Oi 


(1 ~ B) f 
C [l 


a 2 (1 - Ajo 2 
+ (1 + A)(l - B) 


A 2 a 2 L 1 - A 
C V + 1 + A 


_ cfif 2A 2 \ 
“ C + A/ 


In order for this limit to converge to the actual variance a 2 of the function being 
sampled, the relation 

r _ 2 A 2 
1 + A 

must hold. The value of C obtained is inserted into the estimation equation (eq. (4)) to 


& k 2 - B5- k -i 2 + A - * A)( x k - it.,) 5 


DERIVATION OF VARIANCE OF ESTIMATED VARIANCE 


The variance of the estimated variance is also of interest since it gives an indication 
of the error of the estimate. Because of the complexity of the procedure of calculating the 
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variance of the estimated variance for the general case, the derivation is performed here 
only for input data consisting of samples taken from a Gaussian distribution with mean of 
a and variance of o%. However, the technique of estimation described in the previous 
section is not limited to this case; it applies to any probability distribution whose mean 
and variance exist. If the moments of a variable are expressed in terms of the mean and 
variance of the variable, it is found that moments of order greater than two are dependent 
on the probability distribution of the variable. The variance of the estimated variance is 
a function of the probability distribution since it involves moments of order greater than 
two. The moments of a Gaussian variable are used for this derivation. If x is a prob- 
abilistic variable having a Gaussian probability distribution with mean of m and vari- 
ance of v 2 , the first four moments of x are (ref. 5, p. 162): 


*}- 


m 


E 


E <jx 2 j = 

H- 


m 2 + v 2 


E 


H- 


m 2 + 3mv 2 
m^ + 6m 2 v 2 + 3v^ 


Since the equation used for estimation of the mean is a linear equation, the esti- 
mated mean has a Gaussian distribution if the data have a Gaussian distribution. The 
term x^ - x^-i is the difference of two terms, each of which has a Gaussian probability 
distribution. The probability of the difference is also Gaussian. The square of this dif- 
ference has a chi-square distribution with one degree of freedom (ref. 5, pp. 250-253). 

In order to investigate the probability distribution of the estimated variance, it is 
necessary to examine several estimation steps by using 


+ < 1 - B ><‘ ^ % k - Vi f 

The initial guess a Q 2 has a delta function for a probability distribution since it can have 
only one value. The distribution of oq 2 is the weighted convolution of a delta function 
and a chi-square distribution with its origin shifted. The equation for is a weighted 

sum of oq 2 and (x .2 - xq^ 2 . Because of the estimation technique, xq is not indepen- 
dent of oq 2 and is fixed exactly when oq 2 is determined. However, X 2 is independent 
of either oq 2 or xq. The distribution of & 2 2 is a weighted convolution of the chi- 
square distribution representing oq 2 and the distribution pftaj - xq) 2 1 oq 2 J, which is a 
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chi-square distribution with its mean a function of The probability distribution of 

any estimate of the variance by this recursive equation is a weighted convolution of the 
distribution of the previous estimate and a chi-square distribution whose mean is deter- 
mined by the previous estimate of the variance. The probability distribution of the esti- 
mate of the variance is not determined since it is not practical to make a detailed calcula- 
tion. Although the distribution of the estimate of the variance is not derived, its variance 
serves as an indication of the error of the estimate. The error decreases as the vari- 
ance decreases. 

By using the definition of variance, the variance of is 

v[&k 2 } = E{Vj - E 2 {& k 

= E<j Ba k _! 2 + (! - B K1 + A) (x k - Xk-l) 2 ) 2 ! - E 2 ^^ 2 + I L - BK 1 + A ) (x k - 

= b2e j&k-l 4 } + B ( X ' B )(! + A )E^(x k - x k -l) 2 & k -l 2 ] + ^ ~ B)2 4 (1 + — E {( x k " X k-l)^} 

- B 2 E 2 j& k _ 1 2 j - B(1 - B)(l + A)E|& k . 1 2 jE|(x k - x k -l) 2 J 

= B 2 ^^! 4 ! - E 2 j& k _ 1 2 |) + B(1 - B) (1 + A) E jjx k - %_l) 2& k-l 2 } 

" E [(x k - %.i) 2 ]e(\- 1 2 ) + ( - 1: - B)2 4 (1+A)2 E {(*k - x k -l) 4 ] - E2f(x k - x k _l) 2 } 

= fi2v ( & k-l 2 } + B (1 - B )(! + A ) E^k 2 ^.! 2 - 2x k x k _ 1 & k _ 1 2 + 

- B {jx k - Xk.^E^k-i 2 ! + (1 - B ) 2 a + A) 2 v ^ Xk . S k l ) 2 J 

= B 2 V^ k .i 2 ) + (1 - B ) 2 a ± A) 2 V ^(x k - i k _!) 2 j + B(1 - B) (1 + A) (a 2 + o^E^ 2 } 

- 2aE^ k _ 1 a k . 1 2 j + E^k.! 2 ^.^ - E^(x k - S k -i) 2 jE|& k _i 2 j (7) 

This last step can be made since & k _j 2 and x k _j are independent of x k . 
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Let k = n where n is large enough to allow all terms in the equation for V ( crU 

r ^ 


except V^cr^-i J to become infinitesimally close to their limiting values. The conver 

gence of each of these terms is shown by the derivation of their limiting values. See 
appendixes B, C, and D. In appendix B these limits are 

1.-™ i rf/„_ £ . \2~\ _ 8 ct 4 


lim V (x k -x k _^ = 


k^> IT* (1 + A)2 


lim E [(x k - S k .!) 2 } = V 


Appendix C shows that 


lim Ejxk.!^ 2 ] = ao 2 


Appendix D shows that 


lim E fxk-i 2 &k-i 2 "l = a 2 cr 2 + ^ — A + AI — A ^ 1 ~ o’ 4 

k>» l kl kl J [ 1 + A (1 + A) (l - A 2 b) 

By the choice of C in equation (4) , it has been insured that 


Urn E^.! 2 ! = a 2 

Substitution of equations (8) to (12) into equation (9) yields 


v{% 2 } « BZyjVi 2 } 


2\ . (1 - B) 2 (l + A) 2 8a 4 


+ B(1 - B)(l + A)/(a 2 + o- 2 )or 2 - 2a (a a 2 )+ a 2 a 2 + + (1 - A ^(l ~ B ) . o4 - a 2 

+ A) 2 V ' K ' [_ 1 + A d + A)(l - A 2 b)J 1 + A . 


^{^n-l 2 J + 2 d " b ) 2cj4 + B d " B )(l + A) a 2 cr2 + q 4 - 2a 2 cr 2 + a 2 a 2 + ^ ^ " F+” A 

{ 5 n-l 2 } * 2d - B) 2 " 4 + B(1 - B)(l + A)|o4 + ^Ao4 + ^ « 4 - ^ 

f(b„ , 2 \ + 2(1 - B) 2 a 4 + B(1 - B)(l + *. A) 2 (l - B L CT 4 . 


B^h-I 2 } ♦ 2 <1 - B) V ♦ B(1 - B)(l + A)|^ + ' ITA 

= B 2 v[& n .! 2 J + 2(1 - B) V + B(1 \ B)2 £ b AJ - ** 


1 - A 2 B 


H 



A p 

The method of determining the limiting value of variance of a n is to insert some con- 




stant M for V(ct„_i ). Several terms are determined in order to recognize the series 
being generated. Thus, 


"{V! 2 } ■ 


M 


V^& n 2 J = b2m + (1 - B) 2 - 


ML - A ) 2 + 2 

1 - a2 B 


V [ & n + 1 2 ) = b4m + i 1 + b2 )(! " B ) 2 yz ' A 2z + 2 


V^CT n+ 2 2 J = B 6 M + (l + B 2 + B 4 )(l - B) 2 - 


B(1 - A) 2 , 2 

1 - a2 B 


The general term is 


v fa n+ j 2 | = B 2 (j +1 >M + (1 - B) 2 


BU_^2 + 2 


1 - A 2 B 


J 

i=0 


B 


2i 


(14) 


The limiting value is 


lim vfa n+i 2 J = M lim b 2 ^ +1 ^ + (1 - B) 2 

j - 00 l J j-°° 


M^A! 2 „ 2 


1 - a 2 b 


a 4 lim y B 2 * 


j- 00 • 


I 

i=0 


Since 


lim 

j- 00 


I 8 *- ITS* 


i=0 


(IB! < 1.0) 


lim B 2 (i +1 ) = 0 


( I B | < 1.0) 


lim V(a n+j 2 ) = (1 - B) 2 - 


B » - A* 2 + 2 


1 - A^B JVl - B‘ 
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(15) 


j“” V (^ 2 } = *£ V [ 5 n +j 2 } 


_ (1 - g 
V 1 + 3 


B(1 - A) 2 


1 - A 2 B 


+ 2 


No attempt is made to apply the standard mathematical tests for convergence 
because of the complexity of the series. The method of derivation used shows the con- 
vergence of the series since the starting point has no effect on the limiting value of the 
sequence and the limiting value is determined. 

A calculation which adds to the credibility of this derivation is that of the estima- 
tion of the variance when the mean is known exactly. For this case A is equal to 1.0 
and the equation for the variance of the estimated variance reduces to 

lim vfm 2 ) =-J-^|(2a 4 ) (16) 

j oo v*. ^ J 1 + B 

This relation can be checked by actually calculating the mean and variance of the esti- 
mated variance with the mean known exactly. Thus, 

CT k 2 = Baj^j 2 + (1 - B)(x k - a) 2 (17) 


and 


E 




The general term is 


= Ba 0 2 + (1 - B)ct2 
= B 2 a Q 2 + (1 + B)(l - B)ct 2 

E {a 3 2 } = B 3 a Q 2 + (l + B + B 2 )(l - B)o2 



k-1 

= B k a 0 2 + (1 - B)o 2 ^ BJ 
j=0 


(18) 
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Since B < 1.0, 


and 


lim - o 

k— °° 


k-1 

lim Y B-i = 
k— oo 1 " 


Then, 


*. 2 


B 


j=0 


limE(V) = ( 1 -B)a2( f l § ) = a2 


(19) 


The variance of the estimation is determined by 


V< cr 0 ) = 0 


= E^o-! 4 ^ - E 2 


The variances for several values of k are 


vjVJ = (1 - B) 2 (2a 4 ) 


v[& 2 2 ] = 2(1 + B 2 )(l - B ) 2 a 4 


V ( &3 2 } = 2(l + B 2 + B 4 )(l - B) 2 o 4 


The general term is 


k-1 

vf&k 2 ] = 2(1 - B) 2 a 4 ^ B 2 3 

j=0 


The limiting value is 


Hm . 2(1 - B)2c^(l-i p ) . f-|> o* 


( 20 ) 


( 21 ) 


This equation checks with that obtained by letting A = 1.0 in the general equation (15). 
14 



Equation (15) is plotted in figure 1 as a function of A and B. It can be seen from 
figure 1 that the asymptotic value of the variance of the estimated variance can be made 
as small as desired by making B closer to 1.0. The value of A is seen to have very 
little effect on the variance of the estimated variance. Although a mathematical proof is 
not made, it is suspected that as the estimation of the variance is made more accurate 
(A and B near 1.0), the time constant of the estimation is greatly increased. 

CONCLUDING REMARKS 

A recursive equation, which is capable of estimating variance, has been presented 
and analyzed. The constants used in the equation can be varied to control the accuracy of 
the estimation. Additional work is required in this area to determine the effective time 
constant of this estimation technique. 

Langley Research Center, 

National Aeronautics and Space Administration, 

Langley Station, Hampton, Va., June 29, 1969. 
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APPENDIX A 


ANALYSIS OF THE EXPONENTIAL SMOOTHING TECHNIQUE 

This appendix is a derivation of some of the properties of the estimation of the 
mean by the exponential smoothing technique. Although these results were published in 
reference 3 by R. G. Brown, they are derived here in a different manner. 

% = A^k-l + ( X - A ) x k ( k = M,3,- • •) (Al) 


where 


x k 

kth estimate of mean 

x k 

kth data sample 

A 

recursive constant, 

x o 

initial guess of mean 


This equation can be compared with that in reference 3 if (1 - A) is set equal to a. 

Derivation of Mean of Estimation 

The general term of the estimation equation is rearranged by inserting an expres- 
sion for x k _i into the expression for x k , inserting an expression for x k _2, and con- 
tinuing until x 0 is reached. The resulting expression is 

x k = A^q + (1 - A)^x k + Ax k _i + A 2 x k _ 2 + . . . + A k-1 xj) (A2) 

The mean value of x k is 

E (%] = E j Akx o} + (* " A )^ E {xkj + AE^c k _|J + . . . + A k " 1 E^c 1 jj 

The random function from which the samples xj are taken is assumed to be stationary 
with a mean of a and a variance of cr2 so that 
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The mean of x k can be rewritten to yield 


E^x k J = E^A^x^j + (1 - A)E^xjj(l + A + . . . + A k *) 
= A^x 0 + (1 - A)a(l + A + A 2 + . . . + A k_1 ) 


Since | A | < 1.0, 


lim A^Xq = 0 
k— oo 


and 


lim (l + A + A^ + . 

k— co v 




Therefore, 


Lim = 


lim E(x k j = a(l - A) ^ = a 


Derivation of Variance of Estimation 
The variance of x k is calculated by using 


V % = E | x k 2 1 - E2/x k 


The first two terms are 


v{*o) = E {^0 2 ] 


e 2 <M“*o 2 ‘ x o 2 = 0 


and 


(A3) 


(A4) 


(A5) 


(A6) 


ffxij = e|a 2 x 0 2 + 2A(1 - A)x 1 x q + (1 - A) 2 x 1 2 j - E |aXq| + E^(l - A)x|j 

= A 2 x q 2 + 2A(1 - A)ax 0 + (1 - A) 2 (a 2 + a 2 ) - A 2 x Q 2 - 2A(1 - A)ax Q - (1 - A) 2 a 2 


= (1 - A) 2 <x2 


(A7) 
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APPENDIX A 


The variances for several values of k have been calculated by the same technique and 
are presented in the following table: 


k 


0 

0 

1 

(1 - A) 2 o^ 

2 

(l + A 2 )(l - A) 2 *? 2 

3 

(l + A 2 + A 4 )(l - A) 2 *? 2 

4 

(l + A 2 + A 4 + A 6 )(l - A) 2 q2 


The general expression of the 



is written from the table by inspection to yield 



k-l 

(1 . A ) 2 ct 2 £ ( A 2 ) 1 

i=0 


As k approaches infinity, 


(A8) 


lim v(xj\ = (1 - A)2 ct2 — - — 

k— oo J 1 - A2 




1 - A 
1 + A 


a2 


(A9) 


Derivation of Time Constant of Estimation 

Since the estimation equation must also react to step changes in the mean of the 
incoming data, it is desirable to determine the time required to respond to a step change. 
The estimation equation is analyzed as if it were a filter by the use of the z -transform 
method. (See ref. 6.) The impulse response of the following equation is found: 

% = Ax k _i + (1 - A)x k 
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Let 


x 0 = 1 

Xi = 0 (i ^ 0) 

This set of conditions determines the response of the estimation technique to an input of 
a unit impulse at t = 0. From the definition of the z -transform (ref. 6, p. 145) 

OO 

X(z) = ^ x k z _k 
k=0 

X(z) = (1 - A)z^* + A(1 - A)z - -*- + A 2 (l - A)z -2 + . . . 

X(z) = (1 - A)(l + Az" 1 + A 2 z' 2 + . . .) 

X(z) = (1 - A) ^ T 

1 - Az -1 

X(z) = ^ (A10) 

z - A 

The Laplace transform which corresponds to the z -transform is 

H(s) = — ^ (All) 

s - log e A 


The time constant associated with this function is 

t - -T 
c log e A 


(A12) 
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DERIVATION OF THE lim , v f( x k " ^-i) 2 } 


The first moment of (x^ - x^.^) 2 is 


E K x k - x k-l) 2 } = E i x k 2 - 2x k x k-l + x k-l 2 j 


= E 


x k 2 j - 2E { x k} E {^ x k-lJ + E {%-1 2 } 


This step can be made since x^-l and x k are independent. 

lim E^x^ - x^) 2 } = a 2 + a 2 - 2a lim Ej£ k _ij + ^im 

= a 2 + a 2 - 2a(a) + a 2 + ^ ^ cr2 

1 + A 


= 1 + 


1 - A 
1 + 


a 2 


2 a 2 
1 + A 


The second moment is 


E x k - x k-l = E x k 


4 


4x k 3x k- 


6x k 2 %- 


4x k x k 


-l 3 + x k 


4 


where x^ and x^-i both have a Gaussian distribution with known mean and variance 
and are independent. Substitutions of the Gaussian moments yield 
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1“ E § 


lim 
k 


x k - x k 


i) 4 ~\ = a 4 + 6a 2 <j2 + 3 ct 4 - 4a 4 - 12a^a2 + 6a 4 + 6a 2 <j2 + 6 \ ^ a2cr2 
A / J 1 + A 

+ 6 \ ~ — y o 4 - 4a 4 - 12 ^ a2a2 + a 4 + 6 \ - ^ a 2 a 2 + s (\ -- ~ 4 ) a 4 

1 + A 1 + A 1 + A \1 + Ay 


= 3a 4 + 6 ±4 a 4 + 3(^4^) o 4 
1 + A \1 + A/ 



12c 4 
(1 + A) 2 


The limiting value is 


lim V 
k~°° 


(< 


x k ~ x k- 



lim E 

k— oo 


x k " x k- 



lim E 2 
k— oo 




120 4 4 o 4 

(1 + A) 2 (1 + A) 2 

80 4 

(1 + A) 2 
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DERIVATION OF 


lim E, 
k—°° 


% & k 2 


This appendix gives the derivation of 


2 

lim E^xijOjj 

k— oo 


E^Xk&k 2 ) = e|[aVi + (1 - A)x k ] B&k.i 2 + I I. + AK1 - B) ^ _ *^2 1, 

= ABE^-A-l 2 } + B(1 - A)E|ikak.i 2 J 

E {% 2 Vij - 2 E j% x k-i 2 j + E [%-i 3 } 

E f x k 3 J - 2 E { x k\,} + EjxkVl^} 


A(1 + A)(l - B) 


(1 - A)(l + A)(l - B) 


(Cl) 


^ 9 

Since Xk is independent of x k _j and a k _i , the expected values of the product of 
these variables can be separated into the product of the expected values. By using the 
Gaussian moments, equation (Cl) becomes 


E (^ x k & k 2 J = ABE|ick-l&k-l 2 j + B ( J " A )aE|&k_i 2 J + l 1 + A K1 - B) ^ _ A )^ a 3 + 3aa 2) 
+ (3A - 2)^2 + o^Ej^k-lj + (1 - 3A)(a)E|^k-l 2 | + AEj&k-l^j I 


The technique for finding the limit as k approaches infinity of this recursive equa- 
tion is the same as that used in the text of this paper for the variance of the estimated 
variance. The index k is set equal to n where n has a value large enough to permit 

all terms on the right-hand side of equation (C2) except Ej^k-l^k-l^j to become infin- 
itesimally close to their limiting values. By using the Gaussian moments and equa- 
tions (A4) and (A9), equation (C2) becomes 
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E 


jin & n 2 J = ABE|i n _ 1 & n _ 1 2 J + B(1 - A)aa 2 + (1 + A) a - _ B) _ A )(a 3 + 3aa 2 ) 

+ (3A - 2)(a 2 + a 2 )a + (1 - 3A)(a)^a 2 + a 2j + A^a 3 + 3 acr 2 ^ 

= ABEjXn.jan.j^j + (1 - AB)aa 2 


(C3) 


Several terms are calculated in order to recognize the series being generated. Let 

Efxn_ia n _., 2 1 = p 

Then 


and 


‘l^n-l^n-l 2 ! = 

E j^n & n 2 J = AB(P) + (1 - AB)ao 2 
E j^n+l^n+l^J = A 2 B 2 (P) + (1 + AB)(1 - AB)aa 2 

E |^n+2 & n+2 2 J = A 3 B 3 (P) + (l + AB + A 2 B 2 )(1 - AB)ao 2 

E f X n+i & n+i/ = A i+1 B i+1 (P) + (1 - AB)acr 2 Y (AB)j 
^ j=0 


(C4) 


Since A < 1 and B < 1, equation (C4) becomes 


lim E|x n+i a n+i 2 ) = Urn E^ 2 } = !Lr 


= ao 2 


(C5) 


The convergence of this series has been verified by calculating the exact expres- 
sion for the series for k = 0, 1, and 2 but has not been included because of its length. 
From these expressions it is possible to recognize the general expression for the coeffi- 
cients of all terms in the expression. It is found that the limit of coefficients of all terms 
approached zero as k approached infinity except for the coefficient of aa 2 . This coef- 
ficient is found to approach 1.0. 
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DERIVATION OF lim 
k 


This appendix gives the derivation of lim E 


im Ejk k 2 Sj/j 

<0k 2 J> 


] {% 2& k 2 } = 


Ax k _i + 


(1 - A)x k ] 2 [bct^ 2 + (1 + A)(l - B) ^ . 


= A 2 BE|i k _ 1 2 & k _ 1 2 J + 2AB(1 - A)aE^x k _ 1 a k _ 1 2 J + B(1 - A) 2 (a 2 + o^E^.-^ 


(1 + A)(l - B) 


(sfi + 6a 2 o2 + 3 ct^(1 - A) 2 


+ E jk k _jj(a 2 + 3acr2^(-2 + 6 A - 4A 2 ) + Ej£ k _i^j( a2 + cr2^(l - 6A + 6A 2 ) 
+ E|i k _ 1 3 J(a)(2A - 4A 2 ) + E^l^A 2 


(Dl) 


^ ^ 0 

Since x k is independent of x k-1 and o k _j , the expected value of the product of these 
variables is separated into the product of the expected values in this expression. 

The technique for finding the limit as k approaches infinity of this recursive equa- 
tion is the same as that used in appendix C. The index k is set equal to n where n 
has a value large enough to insure that all terms on the right-hand side of equation (Dl) 

except E^ k _j 2 o- k _j 2 j have become infinitesimally close to their limiting values. By 

using the Gaussian moments, equation (Dl) becomes 
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E jx n 2 & n ^j = A 2 BEjfi n _ 1 2 or n _ 1 2 ^ + 2AB(1 - A)a 2 a 2 + B(1 - A) 2 (a 2 + a 2 )cr 2 

+ ^ + ~ + 6a 2 a2 + 3^(1 - A) 2 + (a 4 + 3a 2 cr 2 )^-2 + 6A - 4A 2 ) 

* ~ ^ CT 2 ^a 2 + o' 2 )(l - 6 A + 6 A 2 ) + ^a 2 + 3 j ~ ao^a^A - 4 A 2 ) 


+ la 2 + 


+ 


a 4 + 6 i-^4 a 2 a 2 + 

l + A \1 + Ay 


A 2 


= A 2 BE|i n _i^o- n _i^j + (l - A 2 B)a 2 a 2 


- A)(l - A 2 b) + (1 - A) 2 (l - B) 


(D2) 


The term E jx n _i 2 a n _i 2 J is set equal to an arbitrary constant Q, and several terms 
are calculated in order to recognize the series being generated: 


E ( 5 n-l 2& n-l 2 ) = Q 


E^n 2 ^ 2 ) = A 2 B(Q) + (l - A 2 B)a 2 CT 2 + - A)(l - A 2 b) + (1 - A) 2 (l - B)j 

E j^n+l 2& n+l 2 j = A 4 B 2 (Q) + (l + A 2 B)j^(l - A 2 B)a 2 a 2 

+ TTaE 1 " A)(l - A 2 B) + (1 - A) 2 (1 - B)jj 
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The general term can be recognized to be 


E^ n+j 2 & n+j 2 j = A 2 ^ +1 ^B^ +1 (Q) + ^1 - A2B)a2a2 + ^L[(l - A)(l - A 2 b) 
+ (1 - A) 2 (l - B)]j> £ (A2B) 1 


(D3) 


Since a2b < 1.0, 


Hm E^x k 2 a k 2 j = lim E^x n+j 2 a n+j 2 J 


- ■ A2B ) aV + ttaE 1 - A) (* - a2b ) + (1 - a)2(i - B | 


= a2o2 + 


1 - A , (1 - A) 2 (l - B) 


[l + A (1 + A)(l - a2b) 


(D4) 


The limit of this sequence has been verified by the method of verification discussed 
in appendix C. 
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